Skip to content

Fetch firtool from GitHub - #37

Merged
jackkoenig merged 8 commits into
mainfrom
fetch-firtool-from-github
Nov 17, 2026
Merged

jackkoenig merged 8 commits into
mainfrom
fetch-firtool-from-github

Conversation

@jackkoenig

Copy link
Copy Markdown
Collaborator

Maven Central is cracking down on artifact size so we have to stop repackaging the llvm/circt artifacts as Maven artifacts.

There will be more cleanup to do, but this PR is changing firtool-resolver to fetch artifacts directly from Github releases rather than from Maven. For the time being it will still honor llvm-firtool on the classpath and pull artifacts out of that, but we will drop support for that in version 3.0.

This does change the resolution order in an important way. It adds Java System Property chisel.firtool.version and environment variable CHISEL_FIRTOOL_VERSION which can be used to set the version fetched from Github. There is also chisel.firtool.url and CHISEL_FIRTOOL_URL which is the base URL for downloading artifacts (defaults to https://github.com/llvm/circt/releases/download)1.

The resolution order is now:

  1. Run firtool from the directory specified by CHISEL_FIRTOOL_PATH. An invalid path or a binary that cannot run is an unrecoverable error, as before.
  2. Choose the effective version: system property chisel.firtool.version, then environment variable CHISEL_FIRTOOL_VERSION, then the caller's default version. An explicit override skips the classpath lookup and warns if llvm-firtool is also on the classpath.
  3. If no explicit version is set, honor llvm-firtool on the classpath. This legacy mechanism logs a deprecation message at info level.
  4. Reuse the cached binary at <cache>/<effective-version>/bin/firtool.
  5. Download the effective version from GitHub, verify its checksum, and atomically install just the executable in the cache. Downloads are locked per version across processes.
  6. As a last resort, fetch llvm-firtool with Coursier from Maven. This supports older versions and environments that can reach a Maven proxy but cannot reach GitHub; it only works for already-published artifacts.

This code was developed with AI: Opencode with GPT-6.1 Sol and Claude Opus 5.5.

Footnotes

  1. I don't love that "Chisel" is hardcoded here--I want to change that to be an argument that Chisel passes rather than hardcoded here but I will fix that in a later PR. ↩

@seldridge seldridge left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a deep review.

The overall approach with the choice of fallbacks makes sense to me.

Comment thread .github/workflows/cd-llvm-firtool.yml Resolved
Comment thread .github/workflows/test.yml Resolved
Comment thread .github/workflows/test.yml Resolved
Comment thread README.md Resolved
Comment thread README.md Resolved

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out-of-scope suggestion: This file is getting pretty complicated. I know that LLVM's lit is another dependency (and a Python (!) dependency, too), but that would dramatically clean up the dirtiness here with bash foo.sh | FileCheck. It would also provide parallelism and error reporting.

This is the kind of thing that an AI should be pretty good at refactoring to do. lit is also available via PyPi, so you should be able to grab it from a pipx install.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea, I'll do this in a follow on

Comment on lines +13 to +25
# CHECK: Checking CHISEL_FIRTOOL_PATH for firtool
# CHECK: CHISEL_FIRTOOL_PATH not set
# CHECK: Checking resources for firtool
# CHECK: firtool version not found in resources
# CHECK: Firtool binary with version ([[FIRTOOL_VERSION]]) does not already exist
# CHECK: Attempting to fetch firtool [[FIRTOOL_VERSION]] from GitHub releases
# CHECK: Acquiring firtool cache lock
# CHECK: Downloading SHA-256 checksum
# CHECK: Downloading firtool [[FIRTOOL_VERSION]] for
# CHECK: Extracting firtool from
# CHECK: Verified SHA-256 checksum:
# CHECK: Atomically moving
# CHECK-NOT: Attempting to fetch org.chipsalliance:llvm-firtool:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General comment about all the tests. A lot of this seems like logging information that shouldn't matter exactly as opposed to the final CHECK line which definitely matters. Consider only checking what is really important.

I generally note that this is consistent with the existing testing style, so if you're going to do this, I'd do it in a follow-up.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of it is checking the ordering of things which is intentional (e.g. CHISEL_FIRTOOL_PATH first, then resources, then Github releases, etc.) but some of that could be trimmed down and some of it is not useful (Acquiring firtool cache lock is a implementation detail). I'll look into a larger testing cleanup PR in follow on.

Comment thread firtool-resolver/src/Main.scala Resolved
import TestFixtures._

/** Hermetic resolver tests run real JVMs against local release and Maven fixtures. */
class ResolverTests extends TemporaryDirectorySuite {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never seen TemporaryDirectorySuite before. It might be cleaner to use ScalaTest for this. However, I have no actual objection to this.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was very much the LLM's idea but it's just a very small extension of munit.FunSuite (I am preferring munit to ScalaTest these days, it's a bit more modern and simpler):

From firtool-resolver/test/src/TestFixtures.scala:

** Every test gets its own temporary directory, cleaned up even on failure. */
private[firtoolresolver] trait TemporaryDirectorySuite extends munit.FunSuite {
  protected val directory = FunFixture[Path](
    setup = _ => Files.createTempDirectory("firtool-resolver-tests-"),
    teardown = root => {
      val paths = Files.walk(root)
      try paths.iterator().asScala.toVector.sortBy(_.getNameCount).reverse.foreach(Files.delete(_))
      finally paths.close()
    }
  )
}

Maybe a little bit much but gets the job done.

@jackkoenig
jackkoenig merged commit bdb4636 into main Nov 17, 2026
14 of 15 checks passed
@jackkoenig
jackkoenig deleted the fetch-firtool-from-github branch November 17, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants